Treating Epilepsy by Reinforcement Learning Via Manifold-Based Simulation

The ability to take intelligent actions in real-world domains is a goal of great interest in the machine learning community. Unfortunately, the real-world is filled with systems that can be partially observed but cannot, as yet, be described by first principle models. Moreover, the traditional paradigm of direct interaction with the environment used in reinforcement learning is often prohibitively expensive in practice. An alternative approach simultaneously solves both of these problems by using simulated interaction with the environment rather than real-world experience. The simulation in this approach is a computational model of a dynamical system. The barrier to linking intelligent control with realworld domains is, therefore, one of identifying high-quality state-spaces and transition functions from observations. From a dynamical systems perspective, this barrier is analogous to the problem of finding high-quality manifold embeddings and a rich literature of theory and practice exists to address it. The contribution of this work is two-fold. First, we describe an approach for learning optimal control strategies directly from observations using manifold embeddings as the intermediate state representation. Second, we demonstrate how control strategies constructed in this way can answer important scientific questions. As a concrete example, we use our approach to guide experimental decisions in neurostimulation treatments of epilepsy.